May 15, 2014

POINT OF CONTACT

Principle investigator:
     Kirk L. Clawson
     NOAA Air Resources Laboratory Field Research Division
     1750 Foote Dr.
     Idaho Falls, ID 83402
     Kirk.Clawson@noaa.gov
     (208) 526-2742

README.TXT - Description of Data Files from ARLFRD Sonic Anemometers

Six sonic anemometers were deployed by ARLFRD during PSB1 to measure the turbulence 
field driving the tracer dispersion in the horizontal and vertical. For measurements 
of the vertical turbulence profile on the Grid 3 tall tower (GRI), ARLFRD deployed 
three 3-d sonic anemometers during the study. One Gill Model Solent 1210R3A sonic 
anemometer (designated G1) was installed at 4 m height on a tripod adjacent to GRI. 
One Gill Windmaster Pro sonic anemometer (designated G2) was installed on the tower 
at 30 m height. One R. M. Young Model 81000 Ultrasonic Anemometer (designated R1) was 
installed on the tower at 45 m height. These were complemented by four 3-d Campbell 
Scientific CSAT3 sonic anemometers at 2, 8, 16, and 60 m heights installed by WSU 
(see SonicReadme_WSULAR).

Three of the six 3-d sonic anemometers were R. M. Young Model 81000 Ultrasonic Anemometer 
deployed on the 3200 m arc to evaluate the horizontal homogeneity in the turbulence field 
in conjunction with measurement of turbulence on GRI. All were at 3.2 m height on tripod 
mounts. One anemometer was located near the north end of the 3200 m arc (designated R3), 
one near the south end of the 3200 m arc (designated R4), and one near the 10 m 
meteorological tower (TOW) and ART sodar near the center of the 3200 m arc (designated R2).

Power was supplied to the batteries servicing the sonic anemometer, ART VT-1 sodar, and TOW 
near the center of the 3200 m arc using a large trailer-mounted solar panel array. This 
solar array experienced problems in providing adequate charge to the batteries at all times 
during the experimental period and had largely failed by the end of the experiment. This 
affected data recovery at TOW and the ART but data recovery at R2 was unaffected. The R3 
and R4 sonics near the north and south ends of the 3200 m arc were each supplied with power
by their own solar panel and battery. 

The ARLFRD sonic data were continuously recorded for the duration of the experimental period 
at 10 Hz on a compact flash card inserted into an Acumen Serial Data Collection Bridge. The 
data bridge was setup manually with a laptop computer with the sonic designation at the 
start of its filename (e.g., R1, G1). A GPS unit was also used to verify, and synchronize 
if needed, the correct time in the data bridge.  The sonic data were recorded in an ASCII 
text file. The compact flash cards were gathered and returned to ARLFRD for processing and 
data archival at regular intervals during the 
testing period.

Once the flash cards were returned to ARLFRD, the data was uploaded onto the network for 
processing. The 10 Hz data was parsed into both 10-minute and 30-minute files containing 
roughly 6000 and 18,000 observations, respectively, for the duration of the PSB1 
experimental period. Means, variances, covariances, and other calculations were made on 
these 10 and 30-minute data files. One record was dropped every 26 seconds in R.M. Young 
data records. As a result, the normal 30-minute R.M. Young data file only had about 17930 
records.

The 10-minute averaging time was selected to match with the 10-minute averaging period of 
the SF6 tracer sampling. The longer 30-minute averaging period better accounts for 
nonstationarity effects in the flow and provides a more robust statistical measurement of 
the turbulent fluxes. If necessary, the data was rotated into the correct meteorological 
coordinate system prior to processing (60 degrees, Gill sonic anemometers only).  

Data collected from the six ARLFRD sonic anemometers were subjected to a comprehensive 
quality control and processing software package based upon the schemes detailed in Vickers 
and Mahrt (1997). That included spike detection, coordinate rotations to vertical and 
streamwise mean wind, range checks, amplitude resolution tests, dropout tests, Haar 
transform stationarity (discontinuity) tests, checks for excessive skewness and kurtosis, 
tests for relative systematic flux sampling error (RSET), tests for random flux sampling 
error (RFET), tests for flux variations associated with mesoscale motions (RNT), and tests 
for alongwind relative nonstationarity (RNU), crosswind relative nonstationarity (RNV), 
and vector wind relative nonstationarity (RNS). 

The most common problem with sonic anemometer measurements was spiking in which large, 
random, very brief, and infrequent electronic signal noise is recorded. Spikes were detected 
based upon the criteria of Vickers and Mahrt (1997), with slight modification. This entailed 
identifying 3 or less consecutive points exceeding the range mean  standard deviation for 
a 5-minute (3000 point) moving average. The thresholds used were 4.0 standard deviations for 
u, v, w, and t. For w, the standard deviation threshold was adjusted to 4.5 if it was 
nighttime with temperatures below freezing. This was done to account for nocturnal periods 
often characterized by low and very intermittent turbulence. This did not affect any of the 
data for the IOPs since they were all conducted during the daytime but some nighttime results 
are included in the project database. Spikes were replaced recursively by the mean of the 
nearest non-spike values on either side of the spike. The spike replacement routine was 
repeated for up to 11 passes through the record or until no spikes were detected. The 
threshold increased by 0.1 with each pass. The quality control data files provide information 
on both the total number of passes through the record and the cumulative number of spikes 
detected in all passes. If the total number of spikes detected for any channel exceeded 0.5% 
of the total record on any pass, the record was flagged accordingly. Final calculations were 
done using the despiked output files. These calculations included both the uncorrected and 
corrected mean and flux quantities as well as a suite of quality control parameters. 

The resulting despiked sonic anemometer data sets were plotted and reviewed by the data 
analyst for consistency and accuracy by comparing results with other measurements for the 
duration of each IOP plus one hour before and after each IOP.  This included the following 
comparisons:

*All wind speed and direction measurements in the study area, where available, at 2 m; at 
10 m; at 30 m; at 45 m; and at 60 m. These comparisons included the sonic anemometers, cup 
and vane anemometers at GRI and on the command center meteorological tower (COC), the 10 m 
meteorological tower on the 3200 m arc (TOW), and the ASC and ART sodars. In some cases 
heights were compared if they were close. For example, 2 m cup and vane results at GRI and 
COC were compared with sonic results at 3 m (R2, R3, and R4) and 4 m (G1) and 40 m sonic 
results for ASC and ART were compared to 45 m anemometer results at GRI.
 
*Vertical comparisons of sonic anemometer at 4, 30, and 45 m with cup and vane anemometer 
results at 2, 10, 15, 45, and 60 m on GRI.

The cup and vane anemometers on GRI were calibrated to rigorous standards. The plots of the 
new data sets were reviewed and verified by a second analyst. If any problems or errors were 
discovered, the two analysts had to agree upon and implement a resolution. The initial review 
process discovered a one-hour clock offset error in sonics G1 and G2. The data were adjusted 
accordingly and then there was good agreement with other anemometer data on GRI. Wind speeds 
at the 2, 10, and 30 m heights for COC were also found to be anomalously high in the initial 
review for IOP4. It was found that the maximum 5-minute gusts instead of the mean 5-minute 
wind speeds had been incorporated into the review plots for that IOP. This was corrected and 
then good agreement was found. No other problems or errors were discovered in the measurements 
of wind speed and direction in any of the anemometer data sets. 

There are 12 ARLFRD files containing processed data for the G1, G2, and R1 sonic anemometers 
on GRI. This includes one data file and one quality control file for the 10-minute averaging 
period and one data and quality control file for the 30-minute averaging period for each 
sonic. Similarly, there are 12 ARLFRD data and quality control files containing the processed 
data for the R2, R3, and R4 sonic anemometers located on the 3200 m arc.

The data files for each period are summaries of the measurements and calculated quantities 
during the period. Each file is in CSV format. The files contain data for the month of October, 
when available. The data filenames are specified as XX_PSB1data_##min_October.csv where 
XX is the identity of the sonic anemometer specified in Table 17 (i.e., G1, G2, R1, R2, 
R3, R4) and ## is the averaging period (10 or 30 minutes). The quality control filenames 
are specified as XX_PSB1qc_##min_October.csv. The quality control files contain a listing 
of quality control parameter values and flags for each period. These files encompass the 
testing period of the IOPs with the exceptions already noted (R1).

Potentially spurious data or calculations identified by the flags in columns 4-7 and 24-27 
of the quality control file are automatically designated with -9999' in the corresponding 
processed data files.

The column header designations for the data summary files are:

1. Filename	Start time LLMMDDYYHHMM (LL=location, MM=month, DD=day
                YY=year, HH=hour, MM=minutes (start of period)
2. KNT		Data points in interval
3. VECWD	Vector Wind Direction (despiked)	[degrees azimuth]
4. SCALWS	Mean Scalar Wind Speed (despiked)	[m+1 s-1]
5. SCALWSr	Mean Scalar Wind Speed (raw)		[m+1 s-1]
6. VECWS	Mean Vector Wind Speed (despiked)	[m+1 s-1]
7. USPD_rot	Mean Vector Wind Speed (despiked, rotated/streamwise) [m+1 s-1]
8. VN		Mean north vector			[m+1 s-1]
9. VE		Mean east vector			{m+1 s-1]
10. UVAR	U Variance (despike,detrend,unrotated)	[m+2 s-2]
11. VVAR	V Variance (despike,detrend,unrotated)	[m+2 s-2]
12. WVAR	W Variance (despike,detrend,unrotated)	[m+2 s-2]
13. UVAR_rot	U Variance (despike,detrend,rotated)	[m+2 s-2]
14. VVAR_rot	V Variance (despike,detrend,rotated)	[m+2 s-2]
15. WVAR_rot	W Variance (despike,detrend,rotated)	[m+2 s-2]
16. SIGMAT	Sigma (theta), horizontal		[radians]
17. SIGMAP	Sigma (phi), vertical			[radians]
18. UV_rot	u'v' momentum flux (despike,detrend,rotated)	[m+2 s-2]
19. UW_rot	u'w' momentum flux (despike,detrend,rotated)	[m+2 s-2]
20. VW_rot	v'w' momentum flux (despike,detrend,rotated)	[m+2 s-2]
21. WTBAR_rot	w'T' sensible heat flux (despike,detrend,rotated)[m+1 K+1 s-1]
22. USTR_rot	u* (despike,detrend,rotated)		[m+1 s-1]
23. OLEN_rot	Obukhov Length (despike,detrend,rotated)[m-1]
24. TAVG	Mean Temperature (despiked)		[C]
25. TSDEV	Standard deviation temperature (despike,detrend)[C]
26. UAVGr	Mean U Component Wind Speed (raw)	[m+1 s-1]
27. VAVGr	Mean V Component Wind Speed (raw)	[m+1 s-1]
28. WAVGr	Mean W Component Wind Speed (raw)	[m+1 s-1]
29. USDEVr	Standard Deviation U (raw)		[m+1 s-1]
30. VSDEVr	Standard Deviation V (raw)		[m+1 s-1]
31. WSDEVr	Standard Deviation W (raw)		[m+1 s-1]
32. USTR	u* (despike,detrend,unrotated)		[m+1 s-1]
33. UV		u'v' momentum flux (despike,detrend,unrotated)	[m+2 s-2]
34. UW		u'w' momentum flux (despike,detrend,unrotated)	[m+2 s-2]
35. VW		v'w' momentum flux (despike,detrend,unrotated)	[m+2 s-2]
36. WTBAR	w'T' sensible heat flux (despike,detrend)	[m+1 K+1 s-1]
37. UT		u'T' advective heat flux (despike,detrend)	[m+1 K+1 s-1]
38. OLEN	Obukhov Length (despike,detrend)		[m-1]
39. UAVG	Mean U Component Wind Speed (despiked)	[m+1 s-1]
40. VAVG	Mean V Component Wind Speed (despiked)	[m+1 s-1]
41. WAVG	Mean W Component Wind Speed (despiked)	[m+1 s-1]
42. TAVGr	Mean Temperature (raw)			[C]
43. TSDEVr	Standard deviation temperature (raw)	[C]
44. skwU	Skewness U
45. skwV	Skewness V
46. skwW	Skewness W
47. skwT	Skewness T
48. kurU	Kurtosis U
49. kurV	Kurtosis V
50. kurW	Kurtosis W
51. kurT	Kurtosis T

In the description below, a cycle refers to a single pass through a single record for the 
specified variable during the despiking process. The column headers for the quality control 
parameter file are:

1. XXMMDDYYHRMN where XX is the identity of the sonic anemometer, MM is the month, DD 
is the day, YY is the year, and HRMN is the starting hour and minute of the 10 or 30 
minute averaging period for that row
2. Number of observations in the averaging period
3. Flag=1 if number of observations is more than 100 outside of nominal 10 Hz value for the 
averaging period (10-minutes, 5900-6100; 30-minutes, 17900-18100)
4. Flag=1 if number of spikes in u is greater than 0.5% of observations for any single cycle
5. Flag=1 if number of spikes in v is greater than 0.5% of observations for any single cycle
6. Flag=1 if number of spikes in w is greater than 0.5% of observations for any single cycle
7. Flag=1 if number of spikes in T is greater than 0.5% of observations for any single cycle
8. Total (cumulative) number of spikes detected in u after lpknt_U cycles through record
9. Total (cumulative) number of spikes detected in v after lpknt_V cycles through record
10. Total (cumulative) number of spikes detected in w after lpknt_W cycles through record
11. Total (cumulative) number of spikes detected in T after lpknt_T cycles through record
12. lpknt_U is the number of cycles through u record to eliminate all spikes. The maximum number of cycles allowed is 11.
13. lpknt_V is the number of cycles through v record to eliminate all spikes.
14. lpknt_W is the number of cycles through w record to eliminate all spikes.
15. lpknt_T is the number of cycles through T record to eliminate all spikes.
16. flgRES_U is number of times >70% of bins in 1000 point moving window amplitude resolution test are empty for u
17. flgRES_V is number of times >70% of bins in 1000 point moving window amplitude resolution test are empty for v
18. flgRES_W is number of times >70% of bins in 1000 point moving window amplitude resolution test are empty for w
19. flgRES_T is number of times >70% of bins in 1000 point moving window amplitude resolution test are empty for T
20. flgDRP_U is number of times >15% of points in u record fall in same bin for 1000 point moving window
21. flgDRP_V is number of times >15% of points in v record fall in same bin for 1000 point moving window
22. flgDRP_W is number of times >15% of points in w record fall in same bin for 1000 point moving window
23. flgDRP_T is number of times >15% of points in T record fall in same bin for 1000 point moving window
24. flgABS_U is the number of points in u record > 30 m/s (check after despiking)
25. flgABS_V is the number of points in v record > 30 m/s (check after despiking)
26. flgABS_W is the number of points in w record > |5 m/s| (check after despiking)
27. flgABS_T is the number of points in T record, T > 45C or T<-30C (check after despiking)
28. flgHT1_U number of soft Haar transform threshold exceedances for mean u (2x threshold)
29. flgHT1_V number of soft Haar transform threshold exceedances for mean v (2x threshold)
30. flgHT1_W number of soft Haar transform threshold exceedances for mean w (2x threshold)
31. flgHT1_T number of soft Haar transform threshold exceedances for mean T (2x threshold)
32. flgHT2_U number of soft Haar transform threshold exceedances for standard deviation u (2x threshold)
33. flgHT2_V number of soft Haar transform threshold exceedances for standard deviation v (2x threshold)
34. flgHT2_W number of soft Haar transform threshold exceedances for standard deviation w (2x threshold)
35. flgHT2_T number of soft Haar transform threshold exceedances for standard deviation T (2x threshold)
36. flgHT3_U number of hard Haar transform threshold exceedances for mean u (3x threshold)
37. flgHT3_V number of hard Haar transform threshold exceedances for mean v (3x threshold)
38. flgHT3_W number of hard Haar transform threshold exceedances for mean w (3x threshold)
39. flgHT3_T number of hard Haar transform threshold exceedances for mean T (3x threshold)
40. flgHT4_U number of hard Haar transform threshold exceedances for standard deviation u (3x threshold)
41. flgHT4_V number of hard Haar transform threshold exceedances for standard deviation v (3x threshold)
42. flgHT4_W number of hard Haar transform threshold exceedances for standard deviation w (3x threshold)
43. flgHT4_T number of hard Haar transform threshold exceedances for standard deviation T (3x threshold)
44. flgSKW_U flag=1 for |u skewness| > 1; flag=2 for |u skewness| > 2
45. flgSKW_V flag=1 for |v skewness| > 1; flag=2 for |v skewness| > 2
46. flgSKW_W flag=1 for |w skewness| > 1; flag=2 for |w skewness| > 2
47. flgSKW_T flag=1 for |T skewness| > 1; flag=2 for |T skewness| > 2
48. flgKUR_U flag=1 for u kurtosis < -1 or u kurtosis > 2; flag=2 for u kurtosis < -2 or u kurtosis > 5
49. flgKUR_V flag=1 for v kurtosis < -1 or v kurtosis > 2; flag=2 for v kurtosis < -2 or v kurtosis > 5
50. flgKUR_W flag=1 for w kurtosis < -1 or w kurtosis > 2; flag=2 for w kurtosis < -2 or w kurtosis > 5
51. flgKUR_T flag=1 for T kurtosis < -1 or T kurtosis > 2; flag=2 for T kurtosis < -2 or T kurtosis > 5
52. flgRNU RN alongwind relative nonstationarity test for u; flag=1 for RNU > 0.5
53. flgRNV RN crosswind relative nonstationarity test for v; flag=1 for RNV > 0.5
54. flgRNS RN vector wind relative nonstationarity test for wind speed; flag=1 for RNS > 0.5
55. flgRSET flag=1 for relative systematic flux sampling error test (RSE) > 0.5
56. RSET value for RSE1
57. flgRFET flag=1 for random flux sampling error (RFET) test value > 0.25
58. flgRNT flag=1 for flux trends associated mesoscale motions (RNT) value > 0.25
59. RFET value for RFET1 
60. RNT value for RNT1

Finally, there is a group of text files containing the despiked data for the sonic anemometer 
measurements.  They are reported in a series of data records each 30 minutes in length.  The 
8 half-hour data records covering the two hours during each IOP plus one hour before and after 
are included in the final data set for the project.  The filename convention for the flux 
station sonic tower files is XXMMDDYYHRMN.DSP where the XX specifies the anemometer (e.g., 
G1, R1), MM specifies the month, DD specifies the day, YY specifies the year, HRMN 
specifies the starting time of the half-hour period, and DSP is the extension for the 
despiked files.  These files contain 10 Hz data and have the following columns:

1. U wind component (m/s)
2. V wind component (m/s)
3. W wind component (m/s)
4. Temperature (deg C)

Reference

Vickers, D., and Mahrt, L., (1997), Quality control and flux sampling problems for tower and 
aircraft data. J. Atmos. Oceanic Technol., v. 14, p. 512-526.
